1. Field of the Invention
An object of the present invention is a method for the automatic resetting of images, that can be used notably to display images. It is designed for use chiefly in the medical field where it is sought to compare images representing the same scenes acquired at different times and/or by different protocols. For example, it is sought to put images acquired by different modes, a tomodensitometry mode and an NMR or gammagraphy mode, into correspondence so as to complement the lessons derived from one image with corresponding lessons derived from another image.
In a more common application, the invention is designed to be implemented in angiography where it is sought to depict the structures of vascularization of the tissues. It is recalled that, in angiography, vascular structures are revealed by the comparison of two images acquired respectively with and without the injection of a contrast product into the blood circulation system of a patient. By then subtracting one image from another, point by point, the image of the vascular structures alone is obtained The problem to be resolved, both here and in the comparison of the images acquired with different protocols, lies in the point-by-point comparison. For a great many reasons (essentially related to the shifting of the patient), the structure is not depicted identically from one image to another, and the point-by-point subtraction of the images produces image artefacts at the edges of the tissue structures shown. A shift of about ten pixels should thus be expected when comparing images obtained by different protocols, while a shift of some pixels can be expected in the acquisition of the angiographic images.
For it to be possible to make a subtraction or point-by-point comparison, it is necessary to correct the distortion undergone by one of the two images, called the mask image or image to be reset, with respect to a first image, called a reference image. Once the distortion of the image to be reset has been corrected, the comparison may be done automatically in a simple way. In angiography, for reasons that are intuitively easy to understand, the reference image (which is not made to undergo any correction of distortion) will preferably be the opacified image obtained after injection of the contrast product in the patient's blood. Naturally, the reference image chosen will be the image containing the sensitive information to which particular attention is being paid.
However, the invention may have uses other than those described in the medical field. It may be used, notably, in factory controls where the comparison of images acquired by different protocols (such as X-rays or ultra-sound) is undergoing development. Although the invention is described herein in a medical application, it can clearly be used in these other fields.
2. Description of the Prior Art
The general method for resetting images is known. This method consists of a series of six operations concerning: the pre-processing of the images, the selection of a number of control points (called landmarks), the estimation of the distortion at each of these landmarks, the validation of these landmarks, the interpolation of the distortion to the entire image, and the restoring of the image in corrected form. The pre-processing of the images generally includes at least one digitization of the image so as to replace a physically visible image by a collection of points, also called pixels, the addresses of which are determined and put into correspondence each time with a gray level, or a level of luminosity, assigned to the points.
The selection of the landmarks should meet a certain number of constraints. Notably, the landmarks should be "solid" and distributed homogeneously. A landmark is said to be solid if it is the site of an exact measurement of shift (for a requisite precision). This means that the control points on which, so to speak, the correction of the distortions must be hinged, should be sure points. They should be easily identifiable and unchangeable references. In this sense, the word "landmark", as originally used by sailors in navigation to refer to a rock protruding above sea level, is sufficiently meaningful of their function. Furthermore, the landmarks should be distributed homogeneously. If the landmarks are distributed homogeneously, the modelizing of the distortion will be of good quality at every point of the image. The landmarks should also comply with a constraint of density. The density represents simply the number of landmarks chosen for the image. It will be understood that the greater the number of landmarks, the finer and more precise will be the correction of distortions. However, a large number of landmarks entails, at the same time, a distortion correcting computation which is also heavy. In general, the density is fixed beforehand, depending on an accepted cost of computation.
After the selection of the landmarks in the reference image, a search is made for the position of their corresponding landmarks in the image to be reset. In doing this, a measurement is made of the relative positions between the landmarks and their corresponding landmarks, and the shifts resulting from the distortion are deduced. These shifts are then interpolated to the entire image, and the image to be reset is restored as a function of the interpolated shifts.
An image-resetting scheme such as this has already been proposed. The worth of a method such as this lies in its automatic character. This means that the correction of the images will be done without human intervention. All the work is done by a computer. In practice, automatic methods fail at the stage for selecting and the stage for validating the landmarks. Rather than being performed automatically, these stages of selection and validation are then done manually. In this case, an operator has to intervene to designate the landmarks to be chosen. This manual stage, naturally, has to be ruled out for common use as it calls for a competent operator and requires time which a user often lacks. If the manual intervention is not done, the resetting of the image is poor and, in angiography especially, the images obtained by subtraction then have artefacts at the edges of the structures. The problem to be resolved for the automatic selection and validation of the landmarks is that of choosing only the worthwhile landmarks. The worth or utility of a landmark depends on its solidness. This quality may be expressed in terms of an error EQM attached to each landmark and may be written as follows: EQU EQM=d(V,V)=.vertline.V-V.vertline.
In this expression, V is the true shift, and V is its estimation. For, the true shift V is never known. All that can be done, in seeking the correspondent of a landmark, is to make an estimation, by measurement, of its shifting. And the landmark chosen is all the more worthwhile or useful as this estimation is close to the true shift, namely as EQM is small. We then confront a problem that is difficult to resolve, namely the problem of choosing solid landmarks as a function of their associated true shift (which is never known) and of an estimation of this shift (which is not known for as long as the landmark has not been chosen). The approach wherein this error is computed for all the potential landmarks of the image, that is, ultimately, for all the points of the image, is unusable. It leads to an excessively large mass of computations. For, the computations for estimating the shifts of the landmarks are too lengthy. The approach that has been invented consists, then, in making a prediction of which landmarks will be worthwhile or useful as opposed to the other points of the image which will be rejected.
It is an object of the invention, therefore, to propose a method of prediction of this type. It is possible, in this method, to state, automatically and beforehand, which landmarks will be the most solid and the most homogeneous ones. Essentially in the invention, the solidity of the landmarks is characterized by computations of magnitudes done on the entire image. Using these computed magnitudes, or a combination of them, a prediction is made of the landmarks to be chosen, for which this combination meets a criterion. The principle of the invention, based on the prediction, amounts, then, to the preparing of a tool of measurement, the results of which can be likened to a good prediction. The validity of the method of the invention has been verified.
Furthermore, to guarantee the homogeneity of the distribution of the landmarks, there is a provision in the invention for an arborescent, quadtree type of sub-sampling of the reference image so that, from four neighboring landmarks belonging to four square windows placed in a bigger square, only one landmark representing this square is chosen. Continuing thus, hierarchically, the number of landmarks is reduced while, at the same time, the homogeneity of their distribution is preserved even if, at certain positions of the image, there are landmarks that are less solid than landmarks at other places of the image. In these other places of the reference image, too many landmarks would have been selected if the sole criterion set for selecting them had been their solidity.
Thus, at the time of the operation for the sub-sampling the landmarks, we shall not necessarily take into account the most solid landmark of those in the four windows but, preferably, the landmark with the lowest "cost of transition". The term "cost of transition" is applied to a combination of magnitudes representing, firstly, the solidness of the landmark and, secondly, the place of this landmark with respect to the center of the square containing the four windows to which it belongs. It can be shown that this choice is also such that it will render automatic the prior art manual procedures wherein, at one moment or another, it was necessary to manually re-introduce landmarks at certain important places in the images.